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We show that several versions of Floyd and Rivest's algorithm Select for finding 



, the kih smallest of n elements require at most n + min{A;, n — k} + o{n) comparisons 

' on average and with high probability. This rectifies the analysis of Floyd and Rivest, 

and extends it to the case of nondistinct elements. Our computational results confirm 
. that Select may be the best algorithm in practice. 
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1 Introduction 

^ \ The selection problem is defined as follows: Given a set X := {xj}"^]^ of n elements, a 
CN I total order < on X, and an integer 1 < k < n, find the kth smallest element of X, i.e., 
^ an element x of X for which there are at most k — 1 elements Xi < x and at least k 

Q ■ elements Xj < x. The median of X is the [n/2]th smallest element of X. (Since we are 
c/2 ! not assuming that the elements are distinct, X may be regarded as a multiset). 

^ ' Selection is one of the fundamental problems in computer science. It is used in the 

solution of other basic problems such as sorting and finding convex hulls. Hence its lit- 
^ ' erature is too vast to be reviewed here; see, e.g., |DHUZnit IDoZ99l IDoZOlj and |Knu98| 
■ §5.3.3]. We only stress that most references employ a comparison model (in which a se- 
lection algorithm is charged only for comparisons between pairs of elements), assuming 
that the elements are distinct. Then, in the worst case, selection needs at least (2 + e)n 
comparisons |DoZ01j . whereas the pioneering algorithm of BFP+72] makes at most 5.43r;,, 



its first improvement of |SPP76j needs 3n + o(n), and the most recent improvement in 
[DoZ99j takes 2.9577, + o(n). Thus a gap of almost 50% still remains between the best lower 
and upper bounds in the worst case. 

The average case is better understood. Specifically, for k < [n/2], at least n + k — 2 
comparisons are necessary fCuM89j, |Knu98| Ex. 5.3.3-25], whereas the best upper bound 
is n + k + 0(n^/^ In^*^^ n) Knu98t Eq. (5.3.3.16)]. Yet this bound holds for a hardly 
implementable theoretical scheme |Knu98t Ex. 5.3.3-24], whereas a similar frequently cited 
bound for the algorithm SELECT of |FlR75bj doesn't have a full proof, as noted in |Knu98| 
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Ex. 5.3.3-24] and |PRKT83] . Significantly worse bounds hold for the classical algorithm 
Find of |Hoa61j . also known as quickselect, which partitions X by using the median of a 
random sample of size s > 1. In particular, for k = \n/2] , the upper bound is 3.39n + o{n) 
for s = 1 |Knu98l Ex. 5.2.2-32] and 2.75n + o(n) for s = 3 |(;rii99l IKMP97j . whereas for 
finding an element of random rank, the average cost is 3n + o(n) for s = 1, 2.5n + o{n) for 
s = 3 |KMP97j . and 2n + o{n) when s —>■ oo, s/n — *■ as n — *■ cxo [MaROlJ. In practice 
Find is most popular, because the algorithms of |BFP"'"72l ISPPTGj are much slower on 
the average |Mus971 IVainnj . For the general case of nondistinct elements, little is known 
in theory about these algorithms, but again Find performs well in practice [VainOj . 

Our aim is to rekindle theoretical and practical interest in the algorithm Select of 
FlR75bl §2.1] (the versions of |FlR75b| §2.3] and |FlR75aj will be addressed elsewhere). 



We show that Select performs very well in both theory and practice, even when equal 
elements occur. To outline our contributions in more detail, we recall that Select operates 
as follows. Using a small random sample, two elements u and v almost sure to be just 
below and above the kth are found. The remaining elements are compared with u and v 
to create a small selection problem on the elements between u and v that is quickly solved 
recursively. By taking a random subset as the sample, this approach does well against any 
input ordering, both on average and with high probability. 

First, we revise Select slightly to simplify our analysis. Then, without assuming 
that the elements are distinct, we show that Select needs at most n + mm{k,n — k} + 
O^n^^^ln^^^ n) comparisons on average; this agrees with the result of |FlR,75b[ §2.2] which 
is based on an unproven assumption |PRKT83l §5]. Similar upper bounds are established 
for versions that choose sample sizes as in |FlR75at IMehOOt IRei85j and |MoR95t §3.3]. 
Thus the average costs of these versions reach the lower bounds of 1.5n + o(n) for median 
selection and 1.25n + o(n) for selecting an element of random rank (yet the original sample 
size of |FlR75bt §2.2] has the best lower order term in its cost). We also prove that nonre- 
cursive versions of Select, which employ other selection or sorting algorithms for small 
subproblems, require at most n + min{fc, n — k} + o{n) comparisons with high probability 
(e.g., 1 — 4?T,~^^ for a user-specified (3 > 0); this extends and strengthens the results of 
|C;eS()3l Thm 1], [Mehnni Thm 2] and |MoR95l Thm 3.5]. 

Since theoretical bounds alone needn't convince practitioners (who may worry about 
hidden constants, etc.), a serious effort was made to design a competitive implementation 
of Select. Here, as with Find and quicksort |Sed77j . the partitioning efficiency is crucial. 
In contrast with the observation of |FlR75b| p. 169] that "partitioning X about both u and 
V [is] an inherently inefficient operation" , we introduce a quintary scheme which performs 
well in practice. 

Relative to Find, Select requires only small additional stack space for recursion, 
because sampling without replacement can be done in place. Still, it might seem that 
random sampling needs too much time for random number generation. (Hence several 
popular implementations of Find don't sample randomly, assuming that the input file is 
in random order, whereas others |ValOOj invoke random sampling only when slow progress 
occurs.) Yet our computational experience shows that sampling doesn't hurt even on 
random inputs, and it helps a lot on more difficult inputs (in fact our interest in SELECT 
was sparked by the poor performance of the implementation of |FlR75aj on several inputs 
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of IVainnj ). Most importantly, even for examples with relatively low comparison costs, 
Select beats quite sophisticated implementations of Find by a wide margin, in both 
comparison counts and computing times. To save space, only selected results are reported, 
but our experience on many other inputs was similar. In particular, empirical estimates 
of the constants hidden in our bounds were always quite small. Further, the performance 
of Select is extremely stable across a variety of inputs, even for small input sizes (cf. 
^7.3|) . A theoretical explanation of these features will be undertaken elsewhere. For now, 
our experience supports the claim of |FlR75b| §1] that "the algorithm presented here is 
probably the best practical choice" . 

The paper is organized as follows. A general version of Select is introduced in ^ 
and its basic features are analyzed in ^ The average performance of Select is studied 
in ^ High probability bounds for nonrecursive versions are derived in ^ Partitioning 
schemes are discussed in ^ Finally, our computational results are reported in ^ 

Our notation is fairly standard. \A\ denotes the cardinality of a set A. In a given 
probability space, P is the probability measure, and E is the mean- value operator. 



2 The algorithm Select 

In this section we describe a general version of Select in terms of two auxiliary functions 
s{n) and g{n) (the sample size and rank gap), which will be chosen later. We omit their 
arguments in general, as no confusion can arise. 

Select picks a small random sample S from X and two pivots u and v from 5* such that 
u < xl < V with high probability, where xl is the kth smallest element of X. Partitioning 
X into elements less than u, between u and v, greater than v, and equal to u or v, Select 
either detects that m or f equals xl, or determines a subset X of X and an integer k such 
that xl may be selected recursively as the kth smallest element of X. 

Below is a detailed description of the algorithm. 

Algorithm 2.1. 

Select(X, k) (Selects the kth smallest element of X, with 1 < k < n := \X\) 

Step 1 {Initiation) . If n = 1, return Xi. Choose the sample size s < n—1 and gap g > 0. 

Step 2 {Sample selection). Pick randomly a sample S := {yi, . . . ,ys} from X. 

Step 3 {Pivot selection). Set iu '■= max{ \ks/n — g] , 1}, := min{ \ks/n + g] , s}. Let u 
and V be the iuth and i^th smallest elements of S, found by using Select recursively. 

Step 4 {Partitioning). By comparing each element x of X to m and v, partition X into 
L := {x e X : X < u}, U := {x e X : X = u}, M := {x e X : u < X < v}, 
V := {x & X : X = v}, R := {x & X : v < x}. If A; < n/2, x is compared to v first, and to 
u only if X < f and u < v. If k > n/2, the order of the comparisons is reversed. 

Step 5 {Stopping test). If \L\ < k < \LUU\ then return u; else if \LUUUM\ < k < n-\R\ 
then return v. 

Step 6 {Reduction). If < set X := L and k := k; else ii n — \R\ < k, set X := R 
and k := k — n + \R\; else set X := M and k := k — \L U U\. Set n := |X|. 
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Step 7 (Recursion). Return Select(X, A;). 

A few remarks on the algorithm are in order. 

Remarks 2.2. (a) The correctness and finiteness of Select stem by induction from the 
following observations. The returns of Steps 1 and 5 deliver the desired element. At Step 
6, X and k are chosen so that the kth smallest element of X is the kth smallest element 
of X, and n <n (since u,v ^ X). Also 15*1 < n for the recursive calls at Step 3. 

(b) When Step 5 returns u (or f), SELECT may also return information about the 
positions of the elements of X relative to u (or v). For instance, if X is stored as an array, 
its k smallest elements may be placed first via interchanges at Step 4 (cf. Hence after 
Step 3 finds u, we may remove from 5* its first iu smallest elements before extracting v. 
Further, Step 4 need only compare u and v with the elements of X \ S". 

(c) The following elementary property is needed in ^ Let c„ denote the maximum 
number of comparisons taken by Select on any input of size n. Since Step 3 makes at 
most Cs + Cg-i^ comparisons with s < n, Step 4 needs at most 2(n — s), and Step 7 takes 
at most Cfi with n < n, by induction c„ < oo for all n. 

3 Preliminary analysis 

In this section we analyze general features of sampling used by Select. 

3.1 Sampling deviations and expectation bounds 

Our analysis hinges on the following bound on the tail of the hypergeometric distribution 
established in |Hoe63j and rederived shortly in |Chv79j . 

Fact 3.1. Let s halls he chosen uniformly at random from a set of n balls, of which r 
are red, and r' he the random variable representing the number of red halls drawn. Let 
p := r/n. Then 

P [ r' > ps + ^ ] < e"^^'/" > 0. (3.1) 

We shall also need a simple version of the (left) Chebyshev inequality |Kor78| §2.4.2]. 

Fact 3.2. Let z he a nonnegative random variable such that P[z < C] = 1 fo''^ some 
constant (. Then Ez < t + (P[z > t] for all nonnegative real numbers t. 

3.2 Sample ranks and partitioning efficiency 

Denote by < . . . < x* and yl < ■ ■ ■ < yl the sorted elements of the input set X and 
the sample set S", respectively. Thus x\ is the kth. smallest element of X, whereas u = y*^ 
and V = y*^ at Step 3. This notation facilitates showing that for the bounding indices 

ki := max {\k — 2gn/ s~\ , 1 } and k^. := min { [A; + 2gn/ s~\ , n } , (3-2) 

we have x*j.^ < u < x*i^ < v < with high probability for suitable choices of s and g. 
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Lemma 3.3. (a) P[xl < u] < e ^^^/"^ if iu = \ks/n - ^f] . 

(b) F[u<xl^]< e-2f'/^ 

(c) F[v< xl] < e-^^'/' if = \ks/n + g'\ . 

(d) V[xl^ <v\< e-^'j'l'. 

(e) iu 7^ \ks/n- g\ iff k < gn/s; i^ ^ \ks/n + g] iff n < k + gn/s. 

Proof, (a) If xl < y*^, at least s — + 1 samples satisfy yi > Xj_^i with J := maxj.*=a.* j. 
In the setting of Fact 13. ![ we have r := n — j red elements Xj > Xj_^i, ps = s — js/n 
and r' > s — iu + 1- Since iu = \ks/n — g'\ < ks/n — g + 1 and J > k, we get r' > 
ps + {]- k)s/n + g > ps + g. Hence P[xl < m] < P[r' > ps + g] < e^^f^/'* by ()3.1|) . 

(b) If y*^ < xl^, iu samples are at most x*, where r := maxj.*<a;* j. Thus we have r red 
elements Xj < x*, ps = rs/n and r' > i^- Now, 1 < r < ki — 1 implies 2 < ki = \k — 2gn/s'] 
by ()3.2|) and thus ki < k — 2gn/s + 1, so —rs/n > —ks/n + 2g. Hence iu — ps — g > 
ks/n — g — rs/n — g > 0, i.e., r' > ps + g; invoke ()3.H) as before. 

(c) If y*^ < xl, ijj samples are at most x*., where r := maxj.*<a.* j. Thus we have r red 
elements Xj < x*, ps = rs/n and r' > i^. But i^ — ps — g > ks/n + g — rs/n — g > 
implies r' > ps + g , so again ()3.1|) yields the conclusion. 

(d) If xl < y* , s-iy+l samples are at least x^.^, where J := max3;*=a,* j. Thus we have 
r := ra— Jred elements Xj > Xj_^_i, ps = s — js/n and r' > s — iu + 1- Now, z^, < ks/n + g + 1 
and J > kr > k + 2gn/s (cf fl3.2|) ) yield s — iy + l—ps — g > js/n — ks/n — g — 1 + 1 — g > 0. 
Thus xl^ < V implies r' > ps + g; hence P[xl^ < v] < P[r' > ps + g] < e~^^ by ()3.1|) . 

(e) Follows immediately from the properties of [■] |Knu97l §1.2.4]. D 

We may now estimate the partitioning costs of Step 4. We assume that only necessary 
comparisons are made (but it will be seen that up to s extraneous comparisons may be 
accomodated in our analysis; cf. Rem. Elfa)). 

Lemma 3.4. Let c denote the number of comparisons made at Step 4. Then 

P[c<c]>l- e~2f'/^ and Ec<c + 2{n- s)e-^^^^' with (3.3a) 

c := n + min{ k,n — k} — s + 2gn/s. (3.3b) 

Proof. Consider the event A := {c < c} and its complement A' := {c > c}. U u = v then 
c = n — s < c; hence P[^'] = P[^' fl {n < v}], and we may assume u < v below. 

First, suppose k < n/2. Then c = n — s + \{x ^ X \ S : x < v}\, since n — s 
elements of X \ 5* are compared to v first. In particular, c < 2{n — s). Since k < n/2, 
c = n + k — s + 2gn/s. If t> < xl^, then {x & X \ S : x < v} (Z {x E X : x < v} \ {u, v} 
yields \{x & X \ S : x < v}\ < kr — 2, so c < n — s + kr — 2; since kr < k + 2gn/ s + 1, we 
get c<n + k — s + 2gn/ s — 1 < c. Thus u < v < xl^ implies A. Therefore, A' H {u < v} 
implies {xl^ < v} n {u < v}, so P[A' n {u < v}] < P[xl^ < v] < e-'^^Vs j^Lem. ES^d)). 
Hence we have ()3.3p . since Ec < c + 2(r?, — s)e~^^^/* by Fact 13. 21 f with z := c, ( := 2{n — s)). 

Next, suppose k > n/2. Now c = n — s + \{x E X \ S : u < x}\, since n — s elements 
of X \ S* are compared to u first. If xl^ < u, then {x E X \ S : u < x} <Z {x & X : 
u < x} \ {u,v} yields \{x & X\S : u < x}\ < n — ki — 1; hence ki > k — 2gn/s gives 
c<n — s + {n — k)+ 2gn/ s — 1 < c. Thus A' H {u < v} implies {u < xlj H {u < v}, so 
P[A' n{u< v}] < P[u < x^J < e-'^sVs (^Lgj^_ EHb)), and we get (Q as before. □ 
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The following result will imply that, for suitable choices of s and g, the set X selected 
at Step 6 will be "small enough" with high probability and in expectation; we let X := 
and := if Step 5 returns u or v, but we don't consider this case explicitly. 

Lemma 3.5. F [n < 4:gn/s] > 1 - Ae""^^^^", and En < Agn/s + Ane'^^^^". 

Proof. The first bound yields the second one by Fact 13.21 fwith z := n < n) . In each case 
below, we define an event S that implies the event B := {n < Agn/s}. 

First, consider the middle case of = \ks/n — g] and 2„ = \ks/n + g]. Let £ := {xl < 
u < xl < V < xl^}. By Lem. 13.31 and the Boole-Benferroni inequality, its complement S' 
has F[S'] < Ae-^sVs^ so P[^] > 1 - Ae'^d^/'. By the rules of Steps A-Q, u < xl <v implies 
X = M, whereas xl^ < u < v < xl^ yields h < kr — ki + 1 — 2; since kr < k + 2gn/s + 1 
and ki > k — 2gn/s by ()3.2|1 . we get n < Agn/s. Hence £ C B and thus P[B] > P[£]. 

Next, consider the left case of iu 7^ \ks/n — g], i.e., k < gn/s (Lem. I3.3r e)). If 
7^ \ks/n + (?] , then n < k + gn/s (Lem. I3.3f e)) gives n < n < k + gn/s < 2gn/ S] take 
E := < k + gn/s}, a certain event. For = \ks/n + g], let S := {xl <v< xl^}; again 
F[£]>1- 2e-29'/'' by Lem. |S3i;c,d). Now, xl < v implies X C L U M, whereas v < xl^ 
gives n<kr — l<k + 2gn/s < 3gn/s; therefore S G B. 

Finally, consider the case of 2^, 7^ Iks/n + g], i.e., n < k + gn/s. liiu^ Iks/n — g] 
then k < gn/s gives n < n < 2gn/s; take S := {k < gn/s}. For = \ks/n — g], 
S := {xl^ < u < xl} has P[£] > 1 — 2e~^^^/^ by Lem. I3.3r a.b). Now, u < xl implies 
X C M U -R, whereas xl <u yields n < n — ki with ki > k — 2gn/s and thus n < 3gn/s. 
Hence £ C B. U 

Corollary 3.6. P [c < c and h < Agn/s] > 1 - 4e"^3^/^ 

Proof. Check that S implies A in the proofs of Lems. 13.41 and 13.51 note that n < 2gn/ s 
yields c < 2{n — s) < c (cf. fl3.3b|) ) in the left and right subcases. □ 

Remark 3.7. Suppose Step 3 resets iu '■= iy if k < gn/s, or iy := iu if n < k + gn/s, 
finding a single pivot u = v in these cases. The preceding results remain valid. 

4 Analysis of the recursive version 

In this section we analyze the average performance of Select for various sample sizes. 
4.1 Floyd-Rivest's samples 

For positive constants a and (3, consider choosing s = s{n) and g = g{n) as 

s := min{[a/(?2)],?2 - 1} and 5- := (/5s Inn)^/^ with /(n) := n^/^ In^''^ n. (4.1) 

This form of g gives a probability bound e~^^^/* = for Lems. I3.4H3.5I To get more 
feeling, suppose a = jd = 1 and s = f{n). Let 0(n) := f{n)/n. Then s/n = g/s = (j){n) 
and n/n is at most 40(n) with high probability (at least 1— 4/n^), i.e., 0(n) is a contraction 
factor; note that 0(n) ^ 2.4% for n = 10^ (cf. Tab. lO) . 
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Table 4.1: Sample size f{n) := ri^^^ln^^^ n and relative sample size 0(n) := f{n)/n. 



n 


10^ 


10* 


10^ 


10^ 


5 • 10^ 


10^ 


5 ■ 10^ 


108 


fin) 


190.449 


972.953 


4864.76 


23995.0 


72287.1 


117248 


353885 


568986 


(bin) 


.190449 


.097295 


.048648 


.023995 


.014557 


.011725 


.007078 


.005690 



Theorem 4.1. Let Cnk denote the expected number of comparisons made by SELECT for 
s and g chosen as in ()4.1|) with f3 > 1/6. There exists a positive constant 7 such that 

Cnk<n + mm{k,n-k} + -ffin) yi < k < n. (4.2) 

Proof. We need a few preliminary facts. The function (j)it) := = {\nt/t)^^^ de- 

creases to on [e, 00), whereas fit) grows to infinity on [2, oc). Let 5 := 4(/5/q;)^/^. Pick 
n > 3 large enough so that e — 1 < afin) < n — 1 and e < 5 fin). Let a := a + 1/ fin). 
Then, by ()4.ip and the monotonicity of / and 0, we have for n > n 

s<afin) and /(s) < (4.3) 

fiSfin)) < 6<j>i6fin))fin). (4.4) 
For instance, the first inequality of ()4.3|) yields /(s) < /(a/(n)), whereas 

fiafin)) = a(f)iafin))fin) < a0(a/(n))/(n). 

Also for n > n, we have s = [a/(n)] = afin) + e with e G [0,1) in ()4.ip . Writing 
s = afin) with a := a + e/ fin) G [a, a), we deduce from ()4.1|) that 

gn/s = iP/&f"fin) < iP/af"fin). (4.5) 

In particular, Agn/s < 5 fin), since 5 := 4(/5/a)i/2_ For /5 > 1/6, dHH) implies 

Using the monotonicity of and / on [e, 00), increase n if necessary to get 

2a0(a/(n)) + 50(5/(n)) + 40(^)^^/^-2/3 - ^ q ^^ ^^ 

By Rem. I^Hfc), there is 7 such that ()4.2|) holds for all n < n; increasing 7 if necessary, we 
have 

2a + 25 + 8n^/^-2'^ In"^/^ n < O.O57. (4.8) 

Let n' > n. Assuming ()4.2|) holds for all n < n', for induction let n = n' + 1. 

The cost of Step 3 can be estimated as follows. We may first apply Select recursively 
to S to find u = y*^, and then extract v = y*^ from the elements ■ ■ ■ (assuming 

iu < iv', otherwise v = u). Since s < n', the expected number of comparisons is 

C,,„ + C,_,„,,_,„ < 1.5s + 7/(s) + 1.5(s - «„) + 7/(s - t^) < 3s - 1.5 + 27/(5). (4.9) 
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The partitioning cost of Step 4 is estimated by ()3.3p as 

Ec<n + min{ k,n- k} - s + 2gn/s + 2^^e"^^'/^ (4.10) 

The cost of finishing up at Step 7 is at most C^^ < 1.5n + 7/(17). But by Lem. 13.51 
F[h > 4:gn/s] < 4e"^^^/^ and n < n, so (cf. Fact 13.21 with z := 1.5n + 7/(n)) 

E [ 1.5n + 7/(n) ] < 1.5 ■ Agn/s + -if{Agn/s) + [ 1.5n + 7/(n) ] 46"^^'/". 

Since Agn/s < 6f{n), f is increasing, and /(^) = (j){n)n above, we get 

EC-^, < Qgn/s + -ff{Sf{n)) + [ 1.5 + 70(n) ] 4ne-2»'/^ (4.11) 

Add the costs (gSD, (ICTHl and (ICTll to get 

C„fc < 3s - 1.5 + 27/(5) + n + min{ k,n - k} - s + 2gn/s + 2ne~^'^'^' 
+ Qgn/s + if{6f{n)) + [ 1.5 + 70(n) ] Ane'^^'^' 

<n + mm{k,n- k} + 2s + 8gn/ s + Sne"'^'^^^'' (4.12a) 

+ 7 [ 2/(s) + f{Sf{n)) + 4ne-^^'/'(f){n) ] . (4.12b) 

By fjO|l -(HI)ll. the bracketed term in (I4.12al) is at most 0.057/(n) due to and that 

in ()4.12b|) is at most 0.95/(n) from (j4.7|) : thus (j4.2p holds as required. D 

We now indicate briefly how to adapt the preceding proof to several variations on 1)4.111 : 
choices similar to ()4. 13|1 and ()4.17|1 are used in [MehOOj and |FlR,75aj . respectively. 

Remarks 4.2. (a) Theorem 14. II holds for the following modification of ()4.ip : 

s := min { \af{n)] ,n-l} and g := {ps In 9sY^^ with /(n) := n^/^ In^/^ n, (4.13) 
provided that /3 > 1/4, where 6 > 0. Indeed, the analogue of (031) (cf. (glU), KT^ ) 

gn/s = {P/af/^f{n)i\Yies/\Yinf/^ < {[3 /af'^ f{n)i\Yies/ lYmf'^ (4.14) 
works like ()4.5|) for large n (since lim„_»oo ^^^f = 2/3), whereas replacing ()4.6|) by 

= n{es)-^P < f{n){ae)-^^n^'-"'^/''\n-'^'+"'^/^n, (4.15) 

we may replace n^/^-^z? (tt6')-2^fi(^-'^^)/3 in 

(b) Theorem 14.11 holds for the following modification of ()4.ip : 

s := min { \af{n)] ,n-l} and g := (/3s In'' n)^/^ with f{n) := n^/^ In''/^ n, (4.16) 

provided either = 1 and /3>l/6, ore;>l. Indeed, since ()4.16p = ()4.1|) for 6^ = 1, 
suppose ei > 1. Clearly, hold with 0(t) := f {t)/t. For /3 > 1/6 and n large 

enough, we have g'^/s = jSln'^^n > /31nn; hence, replacing 2(3 by 2/3 and ln~^^'^ by ln~'"'^ 
in ()4.6j) - ()4.8|) . we may use the proof of Thm 14. ll 

(c) Theorem 14. II remains true if we use (3 > 1/6, 

s:=min|an^/^ , n - l} , := (/3s Inn)^/^ and /(n) := n^/^ In^/^ n. (4.17) 

Again (jl31)-(jl3D hold with 0(t) := /(t)/t, and In"^/^ replaces In"^/^ in 

(d) None of these choices gives f{n) better than that in ()4.H) for the bound ()4.2|) . 
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4.2 Reischuk's samples 

For positive constants a and /3, consider using 

1 /2 

s := min { [an^"] , n — 1 } and g:={l3s'rf) with (4.18a) 
Tj := max{ 1 + (e — es)/2,es } < 1 for some fixed < e < e^. (4.18b) 

Theorem 4.3. Let Cnk denote the expected number of comparisons made by Select for 
s and g chosen as in fl4.18|l . There exists a positive constant 'jri such that for all k <n 

Cnfc < + /c, — /;;} + 7,,/,, (t^) with fj^{n):=n'^. (4.19) 

Proof. The function /r,(t) := f grows to oo on (0, cxd), whereas (prjit) := fr]{'t)/'t = t"^^^ 
decreases to 0, so and 0^ may replace / and (p iii the proof of Thm 14.11 Indeed, 
picking n > 1 such that an'^" "£ n — 1, for n > n we may use s = an'^" < afn{n) with 
a < a < a := 1 + 1/n'"' to get anafogues ()4.3|l - ()4.4j) and the following analogue of ()4.5j) 

gn/s = (/?/«)i/2^i+(^-^=)/2 < ^(3/af'^f^{n). (4.20) 

Since g'^/s = (3n^ by ()4.18|1 . and te'^^^yf decreases to for t > := ^ , we may 

replace ()4.6|) by 

ne-^^''' = < n^~"e-^'^''' fr,{n) Wn > n > t^. (4.21) 

Hence, with fi^-v q-'^P^" j-gpiacing n^/^~^^ In"^'''^ n in ()4.7j) - ()4.8|l . the proof goes through. □ 

Remarks 4.4. (a) For a fixed e G (0, 1), minimizing rj in ()4.18|1 yields the optimal sample 
size parameter 

e, :=(2 + e)/3, (4.22) 

with ?7 = > 2/3 and fn{n) = n^^+^^Z^; note that if s = an^" in (j4.18p . then g = {aPY^^n^ii 
with Eg := (1 + 2e)/3. To compare the bounds (j4.2j) and (j4.19|) for this optimal choice, let 
$,(t) := (tyint)i/3, so that $,(t) = f^{t)/f{t) = 0^(t)/0(t). Since lim™<l',(n) = oo, 
the choice ()4.1|) is asymptotically superior to ()4.18|) . However, grows quite slowly, 

and $e(^) < 1 even for fairly large n when e is small (cf. Tab. 14. 2|) . On the other hand, 
for small e and /5 = 1, the probability bound e~^^ = e~^"" of (j4.18j) is weak relative to 

(b) Consider using s := min{ [an""] , n — 1} and g := jS^^'^n''^ with eg, eg G (0, 1) such 
that e := 2eg — > and ri := max{l + tg — e^, e^} < 1. Theorem 14.31 covers this choice. 
Indeed, the equality 1 + — = 1 + (e — es)/2 shows that ()4.18b|) and ()4.20|) remain valid, 
and we have the following analogue of (|4.2ip 

ne-^^"/' < ni"''e-2(/3/-)"7^(n) \/n > n > [{1 - r])a/{2/3e)]'/% (4.23) 

so compatible modifications of (|4.7j) - (j4.8p suffice for the rest of the proof. Note that 
rj > (2 + e)/3 by (a); for the choice = ■^y = jq of |Rei85j . e = | and rj = 
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Table 4.2: Relative sample sizes and probability bounds e 







(tyini) 


1/3 








exp(- 


2n'-) 










n 


10^ 10^ 


5 • 10^ 


10^ 




10^ 




106 


5 


106 




10^ 




1/4 


1.16 1.32 


1.45 


1.52 


3.6 


•10-16 


3.4 


• 10-28 


8.4 


10-42 


1.4 


■ 10- 


49 


e 1/6 


.840 .898 


.946 


.969 


1.2 


• 10-6 


2.1 


• 10-9 


4.4 


10-12 


1.8 


• 10- 


12 


1/9 


.678 .695 


.711 


.719 


7.6 


• 10-4 


9.3 


• 10-5 


1.5 


10-5 


6.2 


• 10- 


6 



4.3 Handling small subfiles 

Since the sampling efficiency decreases when X shrinks, consider the following modifica- 
tion. For a fixed cut-off parameter ricnt > 1, let sSelect(X, /c) be a "small-select" routine 
that finds the fcth smallest element of X in at most Ccut < oo comparisons when |X| < ncut 
(even bubble sort will do). Then Select is modified to start with the following 

Step {Small file case). If n := \X\ < ricnt, return sSelect(X, /c). 

Our preceding results remain valid for this modification. In fact it suffices if Ccut 
bounds the expected number of comparisons of sSelect(X, A;) for n < ricut- For instance, 
()4.2|) holds for n < ricnt and 7 > Ccut, and by induction as in Rem.|2I2tc) we have C„fc < 00 
for all n, which suffices for the proof of Thm 14.11 

Another advantage is that even small ncut (1000 say) limits nicely the stack space for 
recursion. Specifically, the tail recursion of Step 7 is easily ehminated (set X := X , k := k 
and go to Step 0), and the calls of Step 3 deal with subsets whose sizes quickly reach ncut- 
For example, for the choice of ()4.H) with a = 1 and ncut = 600, at most four recursive 
levels occur for n < 2^^ fa 2.15 ■ 10^. 

5 Analysis of nonrecursive versions 

Consider a nonrecursive version of Select in which Steps 3 and 7, instead of Select, em- 
ploy a linear-time routine (e.g., Pick |BFP"'"72] ) that finds the ith smallest of m elements 
in at most 7pm comparisons for some constant 7p > 2. 

Theorem 5.1. Let Cnk denote the number of comparisons made by the nonrecursive ver- 
sion of Select for a given choice of s and g. Suppose s < n — 1. 

(a) For the choice of ()4.H) with f{n) := n^/^ln^''^n, we have 

P [ Cnk < n + min{ k,n — k} + ^pf{n) ] > 1 — 4n~^^ with (5.1a) 
7P := (47P + 2)(/5/a)i/2 + (27P -l)[a + l//(n)] , (5.1b) 
also with f{n) in ()5.1b|) replaced by /(3) > 2 {since n > 3). Moreover, if (3 > 1/6, then 

Ecnk <n + min{ k,n - k} + {^p + A-fp + 2) f{n). (5.2) 

(b) For the choice of (j4.13|) . if 9s < n, then (j5.1a|) holds with n~^^ replaced by 
(Q;6')"2%"^^/^ln"^^/^n. Moreover, if {3 > 1/4, then holds with A'jp + 2 replaced 
by (47p + 2)(a^)-2/3. 



10 



(c) For the choice of ()4.18|) . ()5.1|) holds with f{n) replaced by fqiji) := ri^ and n "^^ 
by e~'^^"'\ Moreover, if n^-v ^-'^f^n" ^ -^^ then ()5.2|) holds with f replaced by 

Proof. The cost Cnk of Steps 3, 4 and 7 is at most 2'~fps + c + 'jpn. By Cor. the event 
C := {c < c,n < Agn/s} has probabihty P[C] > 1 - ie'^^Vs^ jf 

occurs, then 

Cnk < n + min{ k,n — k} — s + 2gn/s + 2'yps + 7p [4:gn/ s\ 

<n + min{ k,n- k} + { A-fp + 2)gn/s+ { 2'^p -l)s. (5.3) 

Similarly, since Ec„fc < 27ps + Ec + 7pEn, Lems. 13. 4H3. 51 yield 

Ec„fc < + min{ fc, n - A; } + ( 47p + 2 ) gn/s + ( 27p - 1 ) s + ( 47p + 2 ) rae"^^'/'. (5.4) 

(a) Since e"^^^/* = n~^^, s = \af{n)'\ < af{n) from s < n — 1 and ()4.3|) . and g'Ti/s is 
bounded by (g^I), (Q implies dSH)). Then follows from and (Q- 

(b) Proceed as for (a), invoking (HTTll - diTni) instead of (ICT and (HH . 

(c) Argue as for (a), using the proof of Thm 14.31 in particular (j4.2(Jj) - (j4.2H) . □ 

Corollary 5.2. The nonrecursive version of Select requires n + min{A;,n — A;} + o(n) 
comparisons with probability at least 1 — An"'^^ for the choice of ()4.1|) . at least 1 — 
4(a6')-2/3n-^^/3 ^/^g c/iozce of and at least 1 - 46"^^"' /or i/ie choice of (ITT^ . 

Remarks 5.3. (a) Suppose Steps 3 and 7 simply sort 5* and X by any algorithm that 
takes at most 75(3^5 + nlnh) comparisons for a constant 75. This cost is at most 
(s + n)75'lnn, because s,n < n, so we may replace 27P by 7s Inn and 47p by 475 Inn in 
flOl- dOjl . and hence in For the choice of (gl]), this yields 

P [ Cnk < n + min{ k,n — k} + 75/(n) In ] > 1 — 4n~^^ with (5.5a) 

75 := (475 + 2 In-i n)(/3/a) + (^^ _ in"! n) [a + l/f{n)] , (5.5b) 

Ec„fc < n + min{ A;,n — /;;} + (75 + 475 + 2 In""*^ n ^ /(n) Inn, (5.6) 

where ln~^ n may be replaced by ln~^ 3, and ()5.6p still needs /5 > 1/6; for the choices ()4.13p 
and (j4.18p . we may modify (j5.5|) - (j5.6p as in Thm l5.lT b.c). Corollary 15 . 21 remains valid. 

(b) The bound (j5.2p holds if Steps 3 and 7 employ a routine (e.g.. Find |Hoa61j . 
AHU74t §3.7]) for which the expected number of comparisons to find the ith smallest of 
m elements is at most 7pm (then Ecnk < 27ps + Ec + 7pEn is bounded as before). 

(c) Suppose Step 6 returns to Step 1 if n > Agnj s. By Cor. 13.61 such loops are finite 
wp 1, and don't occur with high probability, for n large enough. 

(d) Our results improve upon |GeS031 Thm 1], which only gives an estimate like (j5.1ap . 
but with 4n~^'^ replaced by 0(n^~^^/^), a much weaker bound. Further, the approach of 
|GeS03j is restricted to distinct elements. 

We now comment briefiy on the possible use of sampling with replacement. 
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Remarks 5.4. (a) Suppose Step 2 of Select employs sampling with replacement. Since 
the tail bound ()3.1|) remains valid for the binomial distribution |Chv79| IHoe63j , Lemma 
13.31 is not affected. However, when Step 4 no longer skips comparisons with the elements 
of S, -s in dSISI) and is replaced by (cf. the proof of Lem. |S31), 2s in (|4.12a,j) by 

3s and 2a in fl4.8|l by 3a. Similarly, adding s to the right sides of ()5.3j) - ()5.4|l boils down 
to omitting —1 in ()5.1b|) and — ln~^ n in fl5.5b|) . Hence the preceding results remain valid. 

(b) Of course, sampling with replacement needs additional storage for S. This is 
inconvenient for the recursive version, but tolerable for the nonrecursive ones because the 
sample sizes are relatively small (hence ()3.3j) with — s omitted is not too bad). 

(c) Our results improve upon |MoR,95 , Thm 3.5], corresponding to ()4.18|1 with e = 1/4 
and (3=1, where the probability bound 1 — 0(n^^/^) is weaker than our 1 — 4e~^"^''*, 
sampling is done with replacement and the elements are distinct. 

(d) Our results subsume [MehOOt Thm 2], which gives an estimate like ()5.2j) for the 
choice fl4.13|) with (3=1, using quickselect (cf. Rem.EIHtb)) and sampling with replacement 
in the case of distinct elements. 

6 Ternary and quintary partitioning 

In this section we discuss ways of implementing Select when the input set is given as an 
array x[l:n]. We need the following notation to describe its operations in more detail. 

Each stage works with a segment x[l: r] of the input array x[l: n], where 1 < I < r < n 
are such that Xj < Xi for i = 1:Z — 1, Xr < Xi for i = r + l:n, and the kth smallest 
element of x[l:n] is the {k — I + l)th smallest element of a;[/:r]. The task of SELECT is 
extended: given x[l:r] and I < k < r, Select(x, /, r, fc, permutes x[l:r] and finds 

l<k^<k<k+<r such that Xi < Xk for dX\ I < i < Xi = Xk for all k- < i < kj^, 
Xi > Xk for all k+ < i < r. The initial call is SELECT(a;, 1, n, k, /c_, /c+). 

A vector swap denoted by a;[a: b] ^ x[b+l: c] means that the first d := min(6+l— a, c—b) 
elements of array x[a: c] are exchanged with its last d elements in arbitrary order if ci > 0; 
e.g., we may exchange Xa+i ^ Xc-i for < i < d, or Xa+i Xc-d+i+i for < z < 

6.1 Ternary partitions 

For a given pivot v := Xk from the array x[/:r], the following ternary scheme partitions 
the array into three blocks, with Xm < v for I < m < a, Xm = v for a < m < d, Xm > v for 
d < m < r. The basic idea is to work with the five inner parts of the array 



X < V 


X = V 


X < V 


? 


X > V 


X = V 


X > V 


I 


I 


p 


i j 


Q 


f 


r 



(6.1) 



until the middle part is empty or just contains an element equal to the pivot 



X = V 


X < V 


X = V 


X > V 


X = V 


I 


p J 




i q 


f 



(6.2) 
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(i.e., j = i — 1 or j = i — 2), then swap the ends into the middle for the final arrangement 



X < V 

1 



X 



X > V 



d 



(6.3) 



Al. [Initialize.] Set v := Xk and exchange Xi <-> Xk- Set i := / := /, p := / + 1, g := r — 1 
and j := f := r. If f < Xr-, set f := q. If f > Xr, exchange Xi <-h> Xr and set / := p. 

A2. [Increase i until Xi > v.] Increase i by 1; then ii Xi < v, repeat this step. 

A3. [Decrease j until xj < v.] Decrease j by 1; then if Xj > v, repeat this step. 

A4. [Exchange.] (Here xj < v < Xi.) If i < j, exchange Xi ^ xj; then if Xi = v, exchange 
Xi ^ Xp and increase p by 1; if xj = v, exchange Xj ^ Xq and decrease q by 1; return 
to A2. If i = j (so that Xi = xj = v), increase i by 1 and decrease j by 1. 

A5. [Cleanup.] Set a := I + j — p + 1 and d := f — q + i — 1. Exchange x\l: p — 1] ^ x[p: j] 
and x[i: q] <^ x[q + l:f]. 

Step Al ensures that xi < v < Xr, so steps A2 and A3 don't need to test whether i < j; 
thus their loops can run faster than those in the schemes of |BeM93l Prog. 6] and )Knu97| 
Ex. 5.2.2-41] (which do need such tests, since, e.g., there may be no element Xi > v). 



6.2 Preparing for quintary partitions 

At Step 1, r — / + 1 replaces n in finding s and g. At Step 2, it is convenient to place the 
sample in the initial part of x[l: r] by exchanging Xi a;j+rand(r-i) for / < i < := l + s — 1, 
where rand(r — i) denotes a random integer, uniformly distributed between and r — i. 

Step 3 uses ku '■= max{ [/ — 1 + 2s/m — (7] , /} and ky := min{ \l — 1 + is/m + g], Vs} with 
i := A; — / + 1 and m := r — l + 1 for the recursive calls. If SELECT(a;, /, r^, ku, k~ , k^) returns 
ku > ku, we have v := u := Xk^, so we only set k~ := ku, k^ := ku and reset /c^ := ku — 1. 
Otherwise the second call SELECT(a;, ku + 1, Ts, ky, k~, ku) produces v := Xk^. 

After u and v have been found, our array looks as follows 



X < u 


X = u 


U < X < V 


X = V 


X > V 


? 


I 






rvu rvy 




r 



Setting / := k^, p := /c+ + 1, r := r — + q := r — k^ + k^ — 1, we exchange 
x[k^ + 1: Ts] ^ x[rs + 1: r] and then x[k~: k^] ^ x[k'^ + 1: f] to get the arrangement 



X < u 


X = u 


U < X < V 


? 


X = V 


X > V 


I 


I 


p 


K Q 


f 


r 



(6.5) 



The third part above is missing precisely when u = v; in this case ()6.5|) reduces to ()6.H) 
with initial p := p, q := q, i := p — 1 and j '■= q + 1. Hence the case of m = f is handled 
via the ternary partitioning scheme of ^6.11 with step Al omitted. 
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6.3 Quintary partitions 



For the case of /c < [(r + /)/2j and u < v, Step 4 may use the following quintary scheme 
to partition x[l: r] into five blocks, with Xm < u for I < m < a, Xm = u for a < m < b, 
u < Xm < V for b < m < c, Xm = V for c < m < d, Xm > v for d < m < r. The basic idea 
is to work with the six-part array stemming from ()6.5|) 



X = u 


U < X < V 


X < u 


? 


X > V 


X = V 



p 



p 



until i and j cross 



X = u 


U < X < V 


X < u 


X > V 


X = V 


I 


p 


p J 


i q 


f 



we may then swap the second part with the third one to bring it into the middle 



X = u 


X < u 


U < X < V 


X > V 


X = V 



(6.6) 



(6.7) 



(6.8) 



I p b c i q r 

and finally swap the extreme parts with their neighbors to get the desired arrangement 



X < u 


X = u 


U < X < V 


X = V 


X > V 


I 


a 


b c 


d 


f 



(6.9) 



Bl. [Initialize.] Set p := k^^ , q := q, i := p — 1 and j := q + 1. 

B2. [Increase i until Xi > v.] Increase i by 1. If Xi > v, go to B3. If Xi < u, repeat this 
step. (At this point, u < Xi < v.) If Xi > u, exchange Xi ^ Xp] otherwise exchange 
Xi ^ Xp and Xp ^ Xp and increase p by 1. Increase p by 1 and repeat this step. 

B3. [Decrease j until Xj < v.] Decrease j by 1. If Xj > v, repeat this step. If Xj = v, 
exchange Xj ^ Xq, decrease g by 1 and repeat this step. 

B4. [Exchange.] If i > j, go to B5. Exchange Xi ^ Xj. If Xi > u, exchange Xi Xp and 
increase p by 1; otherwise if Xi = u, exchange Xi <-> Xp and Xp Xp and increase p 
and phj 1. If xj = v, exchange xj ^ Xq and decrease g by 1. Return to B2. 

B5. [Cleanup.] Set a := I + i — p, b := a + p — I, d := f — q + j and c := d — f + q. Swap 
x[p: p — 1] x[p: j], x[l: p — 1] x[p: b — 1], and finally x[i: q] ^ x[q + 1: f]. 

For the case of k > [(r + /)/2j and u < v, Step 4 may use the following quintary scheme, 
which is a symmetric version of the preceding one obtained by replacing (|6.6|) - (|6.8|) with 



X = u 


X < u 


? 


X > V 


U < X < V 


X = V 



p 



— •) 

r 



X = u 


X < u 


X > V 


U < X < V 


X = V 


I 


p J 


i q 






f 


X = u 


X < u 


U < X < V 


X > V 


X = V 


I 


p j 


b 


c 


q 


f 



(6.10) 

(6.11) 
(6.12) 
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CI. [Initialize.] Set p '■= p, q '■= q — + + 1, i := p — 1 and j := q + 1, and swap 
x[p: A;" — 1] ^ x[k~: q]. 

C2. [Increase i until Xi > u.] Increase i by 1. If Xi < u, repeat this step. If Xi = u, 
exchange Xi ^ Xp, increase p by 1 and repeat this step. 

C3. [Decrease j until Xj < u.] Decrease j by 1. If Xj < u, go to C4. If xj > v, repeat this 
step. (At this point, u < Xj < v.) If Xj < v, exchange Xj Xg] otherwise exchange 
Xj ^ Xg and Xg ^ Xg and decrease g by 1. Decrease g by 1 and repeat this step. 

C4. [Exchange.] If z > j, go to C5. Exchange Xi xj. If Xi = u, exchange Xi Xp 
and increase p by 1. U xj < v, exchange xj ^ Xg and decrease g by 1; otherwise if 
Xj = V, exchange Xj <-> Xg and Xg Xg and decrease q and g by 1. Return to C2. 

C5. [Cleanup.] Set a := I + i — p, b := a + p — I, d := f — q + j and c := d — f + q. Swap 
x[l: p — 1] ^ x[p: j], x[i: q] ^ x[q + 1: g] and finally x[c + 1: g] ^ x[q + 1: f]. 

To make (jfi.Hj) and ()fi.9|l compatible, the ternary scheme may set b := d + 1, c := a — 1. 
After partitioning / and r are updated by setting I := b if a < k, then I := d + 1 if c < k; 
r := c if k < d, then r := a — 1 if k < b. If I > r, SELECT may return /c_ := k+ := k 
if I = r, k- := r + 1 and k^ := I — 1 if I > r. Otherwise, instead of calling SELECT 
recursively. Step 6 may jump back to Step 1, or Step if sSelect is used (cf. ^4.3^ . 

A simple version of sSelect is obtained if Steps 2 and 3 choose u := v := Xk when 
r — I + 1 < ricut (this choice of |FlR,75aj works well in practice, but more sophisticated 
pivots could be tried); then the ternary partitioning code can be used by sSelect as well. 



7 Experimental results 
7.1 Implemented algorithms 

An implementation of Select was programmed in Fortran 77 and run on a notebook 
PC (Pentium 4M 2 GHz, 768 MB RAM) under MS Windows XP. The input set X was 
specified as a double precision array. For efficiency, the recursion was removed and small 
arrays with n < ricut were handled as if Steps 2 and 3 chose u := v := Xk, the resulting 
version of sSelect (cf. § ^4.31 and l(i.3|l typically required less than 3.5n comparisons. The 
choice of ()4.H] was employed, with the parameters a = 0.5, (3 = 0.25 and ricut = 600 as 
proposed in |FlR75aj : future work should test other sample sizes and parameters. 

For comparisons we developed a Fortran 77 implementation of the riSelect algorithm 
of |ValOOj . Briefly, riSelect behaves like quickselect using the median of the first, middle 



and last elements, these elements being exchanged with randomly chosen ones only if the 
file doesn't shrink sufficiently fast. To ensure 0{n) time in the worst case, riSelect may 
switch to the algorithm of |BFP^72'] , but this never happened in our experiments. 

7.2 Testing examples 

We used minor modifications of the input sequences of |ValOOj . defined as follows: 
random A random permutation of the integers 1 through n. 
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onezero A random permutation of \n/2\ ones and [^/2J zeros, 
sorted The integers 1 through n in increasing order, 
rotated A sorted sequence rotated left once; i.e., (2, 3, ... , n, 1). 
organpipe The integers (1, 2, ... , n/2, n/2, . . . , 2, 1). 

mSkiller Musser's "median-of-3 killer" sequence with n = 4j and k = n/2: 



1 2 3 4 ... k-2k-lkk + l... 2k-2 2k~12k 



twofaced Obtained by randomly permuting the elements of an m3killer sequence in po- 
sitions 4[log2nJ through n/2 — 1 and n/2 + 4[log2riJ — 1 through n — 2. 

For each input sequence, its (lower) median element was selected for k := [n/2]. 

These input sequences were designed to test the performance of selection algorithms 
under a range of conditions. In particular, the onezero sequences represent inputs con- 
taining many duplicates jSed77j . The rotated and organpipe sequences are difficult for 
many implementations of quickselect. The m3 killer and twofaced sequences are hard for 
implementations with median-of-3 pivots (their original versions jMus97j were modified to 
become difficult when the middle element comes from position k instead of A; -|- 1). 

7.3 Computational results 

We varied the input size n from 50,000 to 16,000,000. For the random, onezero and 
twofaced sequences, for each input size, 20 instances were randomly generated; for the 
deterministic sequences, 20 runs were made to measure the solution time. 

The performance of Select on randomly generated inputs is summarized in Table [731 
where the average, maximum and minimum solution times are in milliseconds, and the 
comparison counts are in multiples of n; e.g., column six gives Cavg/n, where Cavg is the 
average number of comparisons made over all instances. Thus 7avg := (Cavg — ^-^n)/ f{n) 
estimates the constant 7 in the bound ()4.2|) : moreover, we have Cavg ~ l-SLavg, where Lavg 
is the average sum of sizes of partitioned arrays. Further, Pavg is the average number of 
Select partitions, whereas iVavg is the average number of calls to sSelect and pavg is the 
average number of sSelect partitions per call; both Pavg and N^^^g grow slowly with Inn. 
Finally, Savg is the average sum of sample sizes; Ss,^^/ f{n) drops from 0.68 for n = 50K to 
0.56 for n = 16M on the random and twofaced inputs, and from 0.57 to 0.52 on the onezero 
inputs, whereas the initial s/f{n) ^ a = 0.5. The average solution times grow linearly 
with n (except for small inputs whose solution times couldn't be measured accurately), 
and the differences between maximum and minimum times are fairly small (and also partly 
due to the operating system). Except for the smallest inputs, the maximum and minimum 
numbers of comparisons are quite close, and Cavg nicely approaches the theoretical lower 
bound of 1.5n; this is reflected in the values of 7avg- Note that the results for the random 
and twofaced sequences are almost identical, whereas the onezero inputs only highlight 
the efficiency of our partitioning. 
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Table 7.1: Performance of Select on randomly generated inputs. 



Sequence 


Size 


Time [msec] 


Comparisons [ 


n] 


7avg 


^avg 


p 

^ avg 


A^avg 


Pavg 






n 


avg 


max 


min 


avg 


max 


min 




[n] 


[In n] 


[In n] 




[%n] 


random 


50K 


3 


10 





1, 


.81 


1, 


.85 


1, 


.77 


5.23 


1.22 


0.46 


1.01 


7.62 


4.11 




lOOK 


4 


10 





1, 


.72 


1 


.76 


1 


.65 


4.50 


1.15 


0.45 


0.99 


8.05 


3.20 




500K 


13 


20 


10 


1, 


.62 


1 


.63 


1 


.60 


4.14 


1.08 


0.59 


1.27 


7.59 


1.86 




IM 


24 


30 


20 


1, 


.59 


1 


.60 


1 


.57 


3.93 


1.06 


0.64 


1.35 


8.18 


1.47 




2M 


46 


50 


40 


1, 


.57 


1 


.58 


1 


.56 


3.73 


1.04 


0.76 


1.59 


7.67 


1.16 




4M 


86 


91 


80 


1, 


.56 


1 


.56 


1 


.55 


3.61 


1.03 


0.94 


1.94 


7.21 


0.91 




8M 


163 


171 


160 


1, 


.54 


1 


.55 


1 


.54 


3.45 


1.03 


0.98 


1.99 


7.45 


0.72 




16M 


316 


321 


310 


1, 


.53 


1 


.54 


1, 


.53 


3.44 


1.02 


0.99 


2.02 


7.55 


0.57 


onezero 


50K 


2 


10 





1, 


.51 


1, 


.52 


1, 


.50 


0.24 


1.02 


0.28 


0.27 


1.17 


3.41 




lOOK 


3 


10 





1, 


.51 


1 


.51 


1, 


.50 


0.23 


1.01 


0.26 


0.25 


1.14 


2.72 




500K 


15 


20 


10 


1, 


.51 


1 


.51 


1, 


.51 


0.26 


1.01 


0.23 


0.23 


1.17 


1.61 




IM 


29 


31 


20 


1, 


.51 


1 


.51 


1, 


.51 


0.26 


1.01 


0.22 


0.22 


1.20 


1.29 




2M 


52 


60 


50 


1, 


.51 


1, 


.51 


1, 


.50 


0.26 


1.01 


0.28 


0.27 


1.14 


1.03 




4M 


110 


111 


110 


1, 


.50 


1, 


.50 


1, 


.50 


0.26 


1.00 


0.33 


0.26 


1.16 


0.83 




8M 


214 


221 


210 


1, 


.50 


1, 


.50 


1, 


.50 


0.26 


1.00 


0.38 


0.25 


1.11 


0.66 




16M 


426 


431 


420 


1, 


.50 


1 


.50 


1 


.50 


0.26 


1.00 


0.36 


0.24 


1.11 


0.53 


twofaced 


50K 


1 


10 





1, 


.80 


1 


.85 


1 


.74 


4.99 


1.21 


0.46 


1.01 


7.53 


4.11 




lOOK 


3 


10 





1, 


.73 


1, 


.76 


1, 


.69 


4.67 


1.16 


0.43 


0.96 


8.23 


3.20 




500K 


13 


21 


10 


1, 


.62 


1 


.63 


1, 


.61 


4.07 


1.08 


0.61 


1.30 


7.85 


1.87 




IM 


24 


31 


20 


1, 


.59 


1, 


.60 


1, 


.58 


3.82 


1.06 


0.67 


1.40 


7.86 


1.47 




2M 


46 


51 


40 


1, 


.57 


1, 


.58 


1, 


.56 


3.66 


1.04 


0.75 


1.58 


7.98 


1.16 




4M 


86 


91 


80 


1, 


.56 


1, 


.56 


1, 


.55 


3.60 


1.03 


0.95 


1.96 


7.36 


0.92 




8M 


164 


171 


160 


1, 


.54 


1, 


.55 


1, 


.54 


3.48 


1.03 


0.96 


1.98 


7.48 


0.72 




16M 


319 


321 


311 


1, 


.53 


1 


.54 


1 


.53 


3.38 


1.02 


1.00 


2.06 


7.74 


0.57 



Table 17^ exhibits similar features of Select on the deterministic inputs. The results 
for the sorted and rotated sequences are almost the same, whereas the solution times on the 
organpipe and mSkiller sequences are between those for the sorted and random sequences. 

The performance of riSelect on the same inputs is described in Tables 17.31 and 17.41 
where N^nd denotes the average number of randomization steps. On the random sequences, 
the expected value of Cavg is of order 2.75n [KMP97], but Table 17^ exhibits significant 
fluctuations in the numbers of comparisons made. The results for the onezero sequences 
confirm that binary partitioning may handle equal keys quite efficiently |Sed77j . The 
results for the twofaced, rotated, organpipe and mSkiller inputs are quite good, since some 
versions of quickselect may behave very poorly on these inputs |Vainnj (note that we used 
the "sorted- median" partitioning variant as suggested in jValOOj ). Finally, the median-of-3 
strategy employed by riSelect really shines on the sorted inputs. 

As always, limited testing doesn't warrant firm conclusions, but a comparison of Se- 
lect and riSelect is in order, especially for the random sequences, which are most 
frequently used in theory and practice for evaluating sorting and selection algorithms. On 
the random inputs, the ratio of the expected numbers of comparisons for riSelect and 
Select is asymptotically 2.75/1.5 ~ 1.83; incidentally, the ratio of their computing times 
approaches 553/316 ~ 1.75 (cf. Tabs. I7H and ITTSj) . Note that Select isn't just asymp- 
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Tabic 7.2: Performance of Select on deterministic inputs. 



Sequence 


Size 


Time [nisec] 


Comparisons [' 


n] 


7avg 




p 

-Tavg 




Pavg 






n 


avg 


max 


min 


avg 


max 


min 




[n] 


[Inn] 


[Inn] 




[%n] 


sorted 


50K 


2 


10 





1 


.80 


1.88 


1 


.71 


4.92 


1.21 


0.44 


0.98 


7.80 


4.08 




lOOK 


2 


10 





1 


.73 


1.76 


1 


.71 


4.76 


1.16 


0.44 


0.97 


7.83 


3.21 




500K 


9 


11 





1 


.62 


1.63 


1 


.61 


4.09 


1.08 


0.60 


1.27 


7.91 


1.86 




IM 


14 


20 


10 


1 


.60 


1.61 


1 


.58 


4.02 


1.06 


0.63 


1.34 


8.05 


1.46 




2M 


25 


30 


20 


1 


.57 


1.58 


1 


.57 


3.75 


1.04 


0.77 


1.60 


7.46 


1.16 




4M 


47 


51 


40 


1 


.56 


1.56 


1 


.55 


3.59 


1.03 


0.95 


1.95 


7.45 


0.91 




8M 


86 


91 


80 


1 


.54 


1.55 


1 


.53 


3.50 


1.03 


0.99 


2.03 


7.55 


0.72 




16M 


160 


161 


160 


1 


.53 


1.54 


1 


.53 


3.37 


1.02 


1.00 


2.04 


7.65 


0.57 


rotated 


50K 


2 


10 





1 


.80 


1.91 


1 


.71 


4.99 


1.21 


0.44 


0.98 


7.90 


4.08 




lOOK 


2 


10 





1 


.74 


1.76 


1 


.70 


4.83 


1.16 


0.44 


0.96 


7.91 


3.21 




500K 


8 


10 





1 


.62 


1.63 


1 


.61 


4.09 


1.08 


0.60 


1.28 


8.01 


1.86 




IM 


14 


20 


10 


1 


.60 


1.60 


1 


.59 


4.03 


1.06 


0.64 


1.35 


8.14 


1.47 




2M 


25 


30 


20 


1 


.57 


1.58 


1 


.56 


3.74 


1.04 


0.76 


1.59 


7.54 


1.16 




4M 


48 


60 


40 


1, 


.56 


1.56 


1 


.55 


3.59 


1.03 


0.94 


1.93 


7.26 


0.91 




8M 


84 


90 


80 


1 


.54 


1.55 


1 


.53 


3.47 


1.03 


0.99 


2.02 


7.43 


0.72 




16M 


161 


171 


151 


1 


.53 


1.54 


1 


.53 


3.35 


1.02 


1.00 


2.04 


7.61 


0.57 


organpipe 


50K 


1 


10 





1 


.80 


1.84 


1 


.70 


5.04 


1.21 


0.46 


1.01 


7.59 


4.11 




lOOK 


2 


11 





1 


.74 


1.76 


1 


.71 


4.88 


1.16 


0.45 


0.98 


8.03 


3.22 




500K 


8 


10 





1 


.62 


1.63 


1 


.60 


4.04 


1.08 


0.62 


1.32 


7.75 


1.87 




IM 


16 


20 


10 


1 


.59 


1.60 


1 


.57 


3.87 


1.06 


0.66 


1.39 


7.72 


1.47 




2M 


30 


40 


20 


1 


.57 


1.58 


1 


.56 


3.69 


1.04 


0.74 


1.56 


7.66 


1.16 




4M 


54 


60 


50 


1 


.56 


1.56 


1 


.55 


3.57 


1.03 


0.97 


1.99 


7.22 


0.92 




8M 


101 


111 


100 


1 


.55 


1.55 


1 


.54 


3.58 


1.03 


0.97 


1.99 


7.38 


0.72 




16M 


194 


201 


190 


1 


.53 


1.54 


1 


.53 


3.39 


1.02 


0.99 


2.02 


7.68 


0.57 


mSkiller 


50K 


2 


11 





1 


.84 


2.27 


1 


.76 


5.61 


1.23 


0.47 


1.04 


7.69 


4.21 




lOOK 


3 


10 





1 


.74 


1.77 


1 


.70 


4.83 


1.16 


0.44 


0.97 


7.79 


3.21 




500K 


9 


10 





1 


.63 


1.64 


1 


.61 


4.24 


1.08 


0.58 


1.23 


7.79 


1.86 




IM 


18 


20 


10 


1 


.59 


1.60 


1 


.58 


3.92 


1.06 


0.67 


1.40 


7.87 


1.47 




2M 


32 


40 


30 


1 


.57 


1.58 


1 


.56 


3.67 


1.04 


0.75 


1.57 


7.85 


1.16 




4M 


57 


61 


50 


1 


.56 


1.56 


1 


.55 


3.64 


1.03 


0.96 


1.96 


7.33 


0.92 




8M 


107 


111 


100 


1 


.54 


1.55 


1 


.54 


3.51 


1.03 


0.96 


1.97 


7.39 


0.72 




16M 


204 


221 


200 


1 


.53 


1.54 


1 


.53 


3.37 


1.02 


0.97 


1.98 


7.64 


0.57 



totically faster; in fact riSelect is about 40% slower even on middle-sized inputs. A 
slow-down of up to 19% is observed on the onezero sequences. The performance gains of 
Select over riSelect are much more pronounced on the remaining inputs, except for 
the sorted sequences on which Select may be twice slower. (However, the sorted input is 
quite special: increasing k by 1 (for the upper median) doubled the solution times of riS- 
elect without influencing those of SELECT; e.g., for n = 16M the respective times were 
169 and 158). Note that, relative to riSelect, the solution times and comparison counts 
of Select are much more stable across all the inputs. This feature may be important in 
applications. 

Acknowledgment. I would like to thank Olgierd Hryniewicz, Roger Koenker, Ronald 
L. Rivest and John D. Valois for useful discussions. 
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Tabic 7.3: Performance of RiSelect on randomly generated inputs. 



Sequence Size 



Time fmsecl 



Comparisons [n] 



P. 



avg 



A^rnd 





n 


avg 


max 


min 


avg 


max 


min 


[Inn] 


[n] 




random 


50K 


2 


10 





3.10 


4.32 


1.88 


3.10 


1.63 


0.45 




lOOK 


4 


10 





2.61 


4.19 


1.77 


2.61 


1.60 


0.20 




500K 


17 


20 


10 


2.91 


4.45 


1.69 


2.91 


1.57 


0.25 




IM 


33 


41 


20 


2.81 


3.79 


1.84 


2.81 


1.57 


0.40 




2M 


62 


90 


40 


2.60 


3.57 


1.83 


2.60 


1.61 


0.35 




4M 


135 


191 


90 


2.86 


4.38 


1.83 


2.86 


1.65 


0.55 




8M 


249 


321 


190 


2.60 


3.48 


1.80 


2.60 


1.58 


0.40 




16M 


553 


762 


331 


2.99 


4.49 


1.73 


2.99 


1.58 


0.40 


onezero 


50K 


1 


10 





2.73 


3.22 


2.68 


2.73 


1.73 


0.00 




lOOK 


3 


10 





2.72 


2.88 


2.68 


2.72 


1.80 


0.00 




500K 


15 


20 


10 


2.74 


2.88 


2.68 


2.74 


1.82 


0.40 




IM 


31 


41 


30 


2.72 


2.85 


2.68 


2.72 


1.84 


0.55 




2M 


62 


70 


60 


2.71 


2.99 


2.68 


2.71 


1.82 


0.75 




4M 


126 


131 


120 


2.73 


2.85 


2.68 


2.73 


1.85 


1.00 




8M 


251 


261 


240 


2.72 


2.88 


2.68 


2.72 


1.87 


1.00 




16M 


505 


521 


491 


2.72 


2.85 


2.68 


2.72 


1.85 


0.95 


twofaced 


50K 


2 


10 





7.77 


8.84 


6.88 


7.77 


1.99 


1.25 




lOOK 


8 


10 





7.76 


9.63 


6.65 


7.76 


2.07 


1.30 




500K 


29 


40 


20 


7.59 


9.09 


6.69 


7.59 


1.91 


1.10 




IM 


58 


70 


50 


7.50 


9.19 


6.63 


7.50 


1.95 


1.30 




2M 


123 


141 


110 


8.07 


9.05 


7.26 


8.07 


2.04 


1.45 




4M 


232 


281 


200 


7.64 


8.86 


6.79 


7.64 


1.93 


1.25 




8M 


458 


530 


401 


7.62 


8.54 


6.96 


7.62 


1.93 


1.35 




16M 


905 


1132 


771 


7.56 


9.10 


6.79 


7.56 


1.94 


1.30 
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Tabic 7.4: Performance of RiSelect on deterministic inputs. 



Sequence 


Size 


Time [msec] 


Comparisons 


[n] 


-^avg 


p 

^ avg 


A^rnd 




n 


avg 


max 


min 


avg 


max 


min 


[Inn] 


[n] 




sorted 


50K 


1 


10 





1.00 


1.00 


1.00 


1.00 


0.09 


0.00 




lOOK 


1 


10 





1.00 


1.00 


1.00 


1.00 


0.09 


0.00 




500K 


4 


10 





1.00 


1.00 


1.00 


1.00 


0.08 


0.00 




IM 


7 


11 





1.00 


1.00 


1.00 


1.00 


0.07 


0.00 




2M 


10 


10 


10 


1.00 


1.00 


1.00 


1.00 


0.07 


0.00 




4M 


22 


30 


20 


1.00 


1.00 


1.00 


1.00 


0.07 


0.00 




8M 


43 


51 


40 


1.00 


1.00 


1.00 


1.00 


0.06 


0.00 




16M 


85 


91 


80 


1.00 


1.00 


1.00 


1.00 


0.06 


0.00 


rotated 


50K 


1 


10 





3.99 


4.04 


3.94 


3.98 


2.32 


1.60 




lOOK 


2 


10 





3.99 


4.03 


3.94 


3.99 


2.28 


1.70 




500K 


10 


10 


10 


3.99 


4.05 


3.95 


3.99 


2.38 


2.15 




IM 


20 


20 


20 


3.98 


4.03 


3.96 


3.98 


2.33 


2.10 




2M 


41 


50 


40 


3.99 


4.05 


3.94 


3.98 


2.36 


2.20 




4M 


83 


90 


80 


3.98 


4.04 


3.96 


3.98 


2.42 


2.70 




8M 


167 


171 


160 


3.99 


4.02 


3.95 


3.99 


2.35 


2.65 




16M 


336 


341 


330 


3.98 


4.02 


3.94 


3.98 


2.37 


2.65 


organpipe 


50K 


1 


10 





8.40 


9.46 


7.00 


8.40 


2.70 


2.95 




lOOK 


6 


10 





8.60 


11.04 


7.35 


8.60 


2.61 


3.20 




500K 


28 


40 


10 


8.51 


11.24 


6.96 


8.51 


2.76 


3.70 




IM 


54 


71 


40 


8.60 


10.62 


7.56 


8.60 


2.87 


4.30 




2M 


109 


131 


90 


8.75 


10.72 


7.69 


8.75 


2.71 


3.95 




4M 


222 


260 


180 


8.94 


10.67 


7.54 


8.94 


2.88 


4.60 




8M 


419 


501 


361 


8.47 


10.22 


7.44 


8.47 


2.82 


4.55 




16M 


862 


1172 


741 


8.71 


11.61 


7.70 


8.71 


2.90 


5.35 


mSkiller 


50K 











8.20 


11.82 


7.01 


8.19 


1.91 


1.55 




lOOK 


5 


11 





8.29 


14.27 


6.91 


8.29 


1.98 


1.50 




500K 


31 


41 


20 


9.52 


14.89 


7.11 


9.52 


1.95 


1.80 




IM 


53 


70 


40 


8.50 


11.77 


7.21 


8.50 


1.81 


1.75 




2M 


104 


140 


90 


8.17 


10.58 


6.92 


8.17 


1.68 


1.80 




4M 


223 


301 


180 


8.99 


12.77 


7.06 


8.99 


1.78 


1.80 




8M 


425 


531 


370 


8.47 


11.16 


7.33 


8.47 


1.69 


1.80 




16M 


840 


1082 


751 


8.31 


11.03 


7.41 


8.31 


1.69 


1.85 
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